🎮
_2 MDX-Net vocal removal model by KimberleyJensen. (_Thanks to runwayml_ space by assets/frameworkHallo: Hierarchical Audio-Driven Visual Synthesis for Portrait Image AnimationShowcasehttps:/github.com/fudan-generative-vision/hallo/assets/17402682/294e78ef-c60d-4c32-8e3c-7f8d6934c6bdFrameworkNews2024/06/15: 🎉🎉🎉 Release the first version on GitHub.2024/06/15: ✨✨✨ Release some images and audios for inference testing on Huggingface.Installation- System requirement: Ubuntu 20.04/Ubuntu 22.04, Cuda 12.1- Tested GPUs: A100conda create -n hallo python=3.10conda activate halloInstall packages with pippip install -r requirements.txtpip install .Besides, ffmpeg is also need:apt-get install ffmpegInferenceThe inference entrypoint script is scripts/inference.py. Before testing your cases, there are two preparations need to be completed:Download pretrained modelsYou can easily get all pretrained models required by inference from our HuggingFace repo.Clone the the pretrained models into ${PROJECT_ROOT}/pretrained_models directory by cmd below:git lfs installgit clone https:/huggingface.co/fudan-generative-ai/hallo pretrained_modelsOr you can download them separately from their source repo:- hallo: Our checkpoints consist of denoising UNet, face locator, image & audio proj.- audio_separator: Kim/_Vocal
🎮

_2 MDX-Net vocal removal model by KimberleyJensen. (_Thanks to runwayml_

by assets/frameworkHallo: Hierarchical Audio-Driven Visual Synthesis for Portrait Image AnimationShowcasehttps:/github.com/fudan-generative-vision/hallo/assets/17402682/294e78ef-c60d-4c32-8e3c-7f8d6934c6bdFrameworkNews2024/06/15: 🎉🎉🎉 Release the first version on GitHub.2024/06/15: ✨✨✨ Release some images and audios for inference testing on Huggingface.Installation- System requirement: Ubuntu 20.04/Ubuntu 22.04, Cuda 12.1- Tested GPUs: A100conda create -n hallo python=3.10conda activate halloInstall packages with pippip install -r requirements.txtpip install .Besides, ffmpeg is also need:apt-get install ffmpegInferenceThe inference entrypoint script is scripts/inference.py. Before testing your cases, there are two preparations need to be completed:Download pretrained modelsYou can easily get all pretrained models required by inference from our HuggingFace repo.Clone the the pretrained models into ${PROJECT_ROOT}/pretrained_models directory by cmd below:git lfs installgit clone https:/huggingface.co/fudan-generative-ai/hallo pretrained_modelsOr you can download them separately from their source repo:- hallo: Our checkpoints consist of denoising UNet, face locator, image & audio proj.- audio_separator: Kim/_Vocal gradio

"Interactive AI demo: assets/frameworkHallo: Hierarchical Audio-Driven Visual Synthesis for Portrait Image AnimationShowcasehttps:/github.com/fudan-generative-vision/hallo/assets/17402682/294e78ef-c60d-4c32-8e3c-7f8d6934c6bdFrameworkNews2024/06/15: 🎉🎉🎉 Release the first version on GitHub.2024/06/1..."

Visit Demo

Best Scenarios

✨ Interactive UI Demo

Technical Constraints

Generic Use
gradio SDK
CPU Config
Running Status
0 Likes

🕸️ Neural Graph Explorer

v15.13

Graph Overview

263 Entities
273 Connections
Explore Full Graph →

📚 Learn More

💻Local Inference

📈 Interest Trend

--

* Real-time activity index across HuggingFace, GitHub and Research citations.

🔬Deep Dive

Expand Details [+]

🛠️ Technical Profile

⚡ Hardware & Scale

SDK
gradio
Hardware
V100
Status
Running

🌐 Cloud & Rights

Source
HuggingFace
License
Open Access

🎮 Demo Preview

Interact with caution. Content generated by third-party code.

💻 Usage

pip install gradio
git clone https://huggingface.co/spaces/assets/frameworkHallo: Hierarchical Audio-Driven Visual Synthesis for Portrait Image AnimationShowcasehttps:/github.com/fudan-generative-vision/hallo/assets/17402682/294e78ef-c60d-4c32-8e3c-7f8d6934c6bdFrameworkNews2024/06/15: 🎉🎉🎉 Release the first version on GitHub.2024/06/15: ✨✨✨ Release some images and audios for inference testing on Huggingface.Installation- System requirement: Ubuntu 20.04/Ubuntu 22.04, Cuda 12.1- Tested GPUs: A100conda create -n hallo python=3.10conda activate halloInstall packages with pippip install -r requirements.txtpip install .Besides, ffmpeg is also need:apt-get install ffmpegInferenceThe inference entrypoint script is scripts/inference.py. Before testing your cases, there are two preparations need to be completed:Download pretrained modelsYou can easily get all pretrained models required by inference from our HuggingFace repo.Clone the the pretrained models into ${PROJECT_ROOT}/pretrained_models directory by cmd below:git lfs installgit clone https:/huggingface.co/fudan-generative-ai/hallo pretrained_modelsOr you can download them separately from their source repo:- hallo: Our checkpoints consist of denoising UNet, face locator, image & audio proj.- audio_separator: Kim/_Vocal/_2 MDX-Net vocal removal model by KimberleyJensen. (_Thanks to runwayml_

Space Overview

Interactive AI demo: assets/frameworkHallo: Hierarchical Audio-Driven Visual Synthesis for Portrait Image AnimationShowcasehttps:/github.com/fudan-generative-vision/hallo/assets/17402682/294e78ef-c60d-4c32-8e3c-7f8d6934c6bdFrameworkNews2024/06/15: 🎉🎉🎉 Release the first version on GitHub.2024/06/15: ✨✨✨ Release some images and audios for inference testing on Huggingface.Installation- System requirement: Ubuntu 20.04/Ubuntu 22.04, Cuda 12.1- Tested GPUs: A100conda create -n hallo python=3.10conda activate halloInstall packages with pippip install -r requirements.txtpip install .Besides, ffmpeg is also need:apt-get install ffmpegInferenceThe inference entrypoint script is scripts/inference.py. Before testing your cases, there are two preparations need to be completed:Download pretrained modelsYou can easily get all pretrained models required by inference from our HuggingFace repo.Clone the the pretrained models into ${PROJECT_ROOT}/pretrained_models directory by cmd below:git lfs installgit clone https:/huggingface.co/fudan-generative-ai/hallo pretrained_modelsOr you can download them separately from their source repo:- hallo: Our checkpoints consist of denoising UNet, face locator, image & audio proj.- audio_separator: Kim/_Vocal/_2 MDX-Net vocal removal model by KimberleyJensen. (_Thanks to runwayml_